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Pj abstract 

We present a panoramic narrow-band study of Ha emitters in the field of the z = 2.16 
proto-cluster around PKS 1138—262 using MOIRCS on the Subaru Telescope. We find 83 
Ha emitters down to a SFR (Ha) ^ IOA/q yr~^ across a ^I'xT region centered on the radio 
galaxy, and identify ^ 10-Mpc scale filaments of emitters running across this region. By ex- 
amining the properties of Ha emitters within the large-scale structure, we find that galaxies 
in the higher-density environments at z = 2.16 tend to have redder colours and higher stellar 
masses compared to galaxies in more underdense regions. We also find a population of Ha 
emitters with red colours {{J — Kg) k, 1), which are much more frequent in the denser envi- 
ronments and which have apparently very high stellar masses with il/* > lO^^il/0, implying 
that these cluster galaxies have already formed a large part of their stellar mass before z 2. 
Spitzer Space Telescope 24/im data suggests that many of these red Ha emitters are bright, 
dusty starbursts (rather than quiescent sources). We also find that the proto-cluster galaxies 
follow the same correlation between SFR and (the "main sequence") of z ^ 2 field star- 
forming galaxies, but with an excess of massive galaxies. These very massive star-forming 
galaxies are not seen in our similar, previous study of z ~ 1 clusters, suggesting that their 
star-formation activity has been shut off at 1 < z < 2. We infer that the massive red (but ac- 
tive) galaxies in this rich proto-cluster are likely to be the products of environmental effects, 
and they represent the accelerated galaxy formation and evolution in a biased high density 
region in the early Universe. 

Key words: galaxies: clusters: individual: PKS 1138—262 — galaxies: evolution — large- 
scale structure of Universe. 



1 INTRODUCTION 



It is widely recognized that galaxy properties such as star- 
formation rate (SFR) and morpholo gy depend on t h e environment 
in wh i ch t he galaxies res ide (e.g. Dresslei 1980l: iKodama e t all 
200 ll: [Lewis et al 200^ lOomez et alj l2003l : lOoto et al. 200: 



TanakaetalJ [20041) . Local galaxy clusters are dominated by red 



quiescent ellipticals and SOs, and their tight colour-magnitude re- 
latio n suggests very old stellar populations in these cluster galaxies 



Bower. Lucey & Ellisll 19921 : iKodama et al]| 19981 : [Smith et al.l 



[2012h . This means that many present-day cluster galaxies must 
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have formed their stars in the early Universe at z 2> 1, and there- 
fore distant clusters of galaxies are an ideal site for studying this 
early evolutionary stage of cluster galaxies. 

Earlier studies have identified a high fraction of star- 
formi ng population in distant clusters (e.g. [Butcher & Oemle3 
In addition to this blue star-forming population, opti- 
cally red star-forming galaxies, as well as a starbursting pop- 
ulation revealed by mid-infrared (MIR) and radio observations 
are also reported to i ncrease in frequency in distant clusters 
(e.g. [Smail et al.[[l999[: [Oeach et al.[ [2006[: [Marcillac et alj[2007l : 



[Saintonge. Tran. & HolderJ 12008': 'Haines et al."2009^. particularly 
in the cluster outskirts (e.g. Koyama et al., 2008 : 2011). In the 
very rich cluster cores, however, the star-forming galaxy fraction 
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is still very low up to 2: ^1 (e.g.|C ouch et al. 2001 ; Baloah e t alj 
2OO2I: iKodama et al.ll2004l : iKovama et al. 2010; Sobral et al. 2oTlF 



Bauer et al.ll201lh rsuggesting that the major phase of star forma- 



tion for galaxies in these regions must be occurring at an even 
earlier epoch. Indeed some recent studies have also suggested 
a "reversal" of star formation-den sity relation at 2 ~ 1 (e.g. 
lElbaz et al .l2007l : ICooper et alj2008l) in high-density environments 
in the distant Universe. Therefore, the next challenge must be to 
study directly the site of cluster galaxy formation al z ^ 1. New 
surveys have unveiled several clusters of galaxies at these early 
times, z ^ 1.5, using a var i ety of techniques (e.g. Stanford et al.l 
20061: lAndreori et al.l |2009|: iTanaka. Finoguenov. & Uedd I2OI0I : 



Papovich et aljboiol : I Gobat et al.ll201 ih . and these are good lab- 
oratories for studying th e early activity of cl uster galaxies (e.g. 
l^uadrTeTal. 2012; Raic hoor & Andreonll2012l) . By conducting an 
[On] emission -line survey o f a z = 1.46 cluster with the Sub- 
aru Telescope, iHayashi et all l l2010h first noted a large fraction of 
[On] galaxies in the cluster, and they highlighted the high activity 
of cluster core galaxies at z ~ 1.5. Subsequent studies of simi- 
larly high-redshift clusters also con firm active galaxy populations 
in these high-density environments jTran et al.ll201(]| ; iHilton et al.l 
l20ld ; lFassbenderetai]|201ll ; iTadaki et al.ll2012l) . suggesting we 
are approaching the formation epoch of present-day cluster galax- 
ies. 

It is still challenging to detect over-dense regions in the more 
distant Universe at z Z 2 (i.e. the p eak epoch of galaxy activ - 
ity in the general field population; e.g. lHopkins & Beacomll200^ . 
Bright high-redshift radio galaxies or quasors (QSOs), or indeed 
overdensities comprising several QSOs, have been used as land- 
marks of high-redshift over-dense regions. This approach has been 
successful in the la st decade, and many "proto-clusters" have 
been iden tified (e.g. 'Ov erzier et al.l 1 20061 ; IVenemans et al.l 1 20071 ; 
iMilev &"D e Breuck 2008 and references therein). One of the most 
effective ways to select "member" galaxies associated to the proto- 
clusters is to exploit narrow-band filters to pick up strong emission- 
lines (such as Lyg, [Oil] or Hq) at the par ticular redshifts (e.g . 
Steidel et al."2000'; i Kurk et al.ll2000l ; 2004a; iMatsuda et al.|[2oTll ; 
Yamada et al. 2012). Considering the importance of dust-obscured 
activity in the early Universe as stated above, the Ha line is the 
preferred tracer because it is less affected by dust extinction (or 
metallicity) compared to other strong lines emitted in the rest-frame 
optical or ultra-violet (UV), which tend to be biased to lower- 
metallicity galaxies with little dust extinction. An observational 
challenge is that the Ha lines of distant galaxies (at 2: Z 0.5) shift to 
near-infrared regime (NIR), where the night sky emission is strong, 
and where efficient large format of detectors became available only 
recently. 

The PKS 1138—262 field is one of the best-studied proto- 
clusters, centered on a radio galaxy al z — 2.156, and impor- 
tantly, the Ha line from this redshift falls in between the night 
sky emission lines in the NIR and c an be observed with ground- 
based telescopes l lKurketal.|[2004j b). The central radio galaxy 
(PKS 1138) is one of the brightest radio ga laxies known at 2 ^ 2 
with a clumpy morphology in the optic al ([Pentericci et alj[l997l ; 
1998) and distorted morphology in radio jCarilli et al.l 19971 ; 2002). 
The pioneering Lya imag ing and spectrosco py a round this radio 
alaxy was performed by iKurk et al.l ( I2OOOI) and IPentericci et al.l 
adies spectroscopically confi rmed 14 Lya emitting 
galaxies at 2 ~ 2.16. IPentericci et alj ( I2OO2I) used X-ray observa- 
tions to identify 18 X-ray sources in the PKS 1138 field, among 
which five were confirmed to be cl uster members by subsequent 
spectroscopic campaign of this field jKurk et al.l2004bl ; ICroft et"ail 



I2OO5I) . An intensive search for Ha emitters and extremely red ob- 
jects (EROs) was also performed bv iKurk et al.i t2004ai) with VLT. 
They reported ~ 40 Ha emitter candidates within a ~ 12 arcmin^ 
field, 9 of them being spectroscopically confirmed ( 3 have de- 
tectabl e Ha+[Nll] emission in their NIR spectra; see iKurk et al.l 
for details). Following these studies, more recent work has 



cluster (Milev et al.ll2006l; Kodama et alj 20071; Zirm et al. 


2008 


Hatch et alj 


20081; 2009; iDohertv et alj|2010l; iTanaka et al 


201c 


Kuiper et al 


2011b. Given the strong evidence of the over-densitv 



of galaxies, this PKS 1 138 field may be an ideal laboratory for test- 
ing the environmen tal dependence of galaxy properties at 2 ~ 2 
( lTanakaetai]|2010l) . 

In this paper, we revisit the PKS 1138 field using observations 
from the wide-field NIR camera, MOIRCS (Ichikawa et al. 200^ 
ISuzuki et al]|2008l) on the Subaru Telescope dive et al.ll2004l) . Our 
aim is to map the entire structure around this proto-cluster and 
the environmental dependence of Ha-based star forming activity 
at 2 ~ 2 within this structure. We show our Subaru data and sup- 
plementary data in §2. The Ha emitter selection procedure and the 
derivation of physical quantities are presented in §3. Our main re- 
sults and discussions are shown in §4, and we give a summary of 
this paper in §5. Throughout this paper, we adopt the standard cos- 
mology with = 0.3, = 0.7, and Ho = 70kms"^ Mpc"\ 
which gives a 1" scale of 8.29 kpc at the redshift of the PKS 1138: 
2 — 2.156. Magnitudes are given in the AB system, unless other- 
wise stated. 



2 DATA 

2.1 MOIRCS data 

We are undertaking a long-term study of galaxies in dense environ- 
ments at high redshifts: MApping HAJpha an d Lines of Oxygen wit h 
Subaru (MAHALO-Subaru; see overview bv lKodamaetalJl2012h . 
As a part of this project, we observed the field around the 2 = 
2.156 radio galaxy PKS 1 138 through the J, Ks, and NB2071 fil- 
ters using the MOIRCS imaging spectrograph on the Subaru Tele- 
scope that has a 4'x7' field. The NB2071 filter (Ac = 2.068^tm, 
A A = 0.027^im) covers the HaA6563 line at 2 ~ 2.13-2.17, cor- 
responding to velocities of —2500 % An ^ 1500 km s~^ relative 
to the radio galaxy (see Fig. [TJ. Note that the NB2071 filter was 
fabricated for these specific observations, and therefore this filter 
perfectly matches to the redshift distribution of spectroscopically 
confirmed member galaxies in the stru cture around PKS 1 138 (see 
Fig. [TJ. Based on the known redshifts l lKurk et al]|2004bh . we ex- 
pect to be able to detect > 90% of the Ha emitting cluster members 
with this filter. 

We observe two pointings with MOIRCS, PKS 1138-C and 
PKS 1138-S (each 4'x7' in extent). The PKS 1138-C field (R.A. 
ll''40™48".4, Dec. -26'^29™09=.0, J2000) is centered on the ra- 
dio galaxy, and the PKS 1138-S field centre (R.A. ll'M0™48'.4, 
Dec. -26'^32™44=.0, J2000) is ~ 3.5' away from the radio galaxy 
towards the south. Note that this Ha emitter survey covers a fac- 
tor of ~ 4 X lar ger area than the pr evious Ha survey with ISAAC 
on the VUT by Kurk et al] ( l2004ah . and this is indeed one of the 
widest-field Ha survey targeting a high-redshift structure to date. 
The observation were carried out under good conditions (see the 
data summary in Table 1), and the data were reduced in a standard 
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Figure 1. The velocity distribution (relative to the radio galaxy) of the spec- 
troscopically confirmed proto-cluster member galaxies shown by Pentericci 
et al. (2000), Kurk et al. (2004b), and Doherty et al. (2010) (solid-line his- 
togram), with an arbitrarily scaled transmission curve of the NB2071 filter. 
We also show the velocity distribution of the emission-line sources close 
to the radio galaxy (within ~15"x 15") revealed with integral field spec- 
troscopy by Kuiper et al. (201 1) (dotted-line histogram). 



manner with MCSRED0 s oftware, which was written by one of the 
co-authors (I. Tanaka; see lTanaka et al.ll201 ih . We use the NBMC- 
SALL package in MCSRED for narrow-band (NB) data reduction, 
which was designed to reduce MOIRCS NB data. We also use the 
J- and Ks-hwA i maging data of the PK S 1 138-C field obtained in 
2006 presented bv lKodama et al.l jlOOTl) . We mosaiced all the data 
together, and the final image size is 6.7' x 7.5'. Note that all the data 
are smoothed to 0.7" FWHM seeing before measuring the multi- 
band photometry, and all the analyses in this paper were performed 
on these smoothed images. We estimated the limiting magnitudes 
of each image by measuring the variance of 1.5" diameter aperture 
photometry at random positions on each image. The data quality 
is effectively uniform, but the limiting magnitudes differ slightly 
between the PKS 1 138-C and PKS 1138-S fields, due to the small 
differences in their total exposure times (see Table 1). 



2.2 Optical data 

We obtained a new deep ^'-band image with Suprime-Cam on the 
Subaru Telescope. This observation was executed on 29th April 
2011 under photometric conditions (see Table 1). We r educed the 
data in a standard mann er with the SDFRED2 software jYagi et al] 
I2OO2I : lOuchi et al.ll2004h . Note that the field of view of Suprime- 
Cam (34' X 27') is much larger than our MOIRCS data coverage, 
and so we trimmed the 2;' -band image accordingly. The limiting 
magnitude was estimated in the same way as for the MOIRCS data. 

We al s o use the B-band data taken with FORS 1 on VLT by 
iKurk et al.l ( I2OOOI) . Note that the B-band data and our MOIRCS 
data do not completely overlap with each other. In addition the ex- 
posure time of this B-band data is only 30 minutes. As a result the 
limiting magnitude of the B-band data, estimated in the same man- 
ner as for the Subaru data, is 26.2 mag which is not deep enough for 



http://www.naoj.org/staff/ichi/MCSRED/mcsred_e.html 



studying passive z ~ 2 galaxies. But we can still use the B-band 
data to check the BzK diagram for narrow-band sources i n §3.2. The 
seeing measured from the B-band image is ~ 0.7" (see lKurk et al.l 
l2004ai) . 



2.3 Photometric catalogue 

We construct a photometric catalogue of our data usi ng the SEx- 
TRACTOR software package jSertin & Arnoutsiri996l) . The source 
extraction is based on the NB2071 image, and we use the double- 
image mode of S EXTRACTOR to perform multi-band photome- 
try. After removing saturated sources, the catalogue includes 1038 
sources with > 5cr detection in NB2071 (i.e. mNB207i < 22.9). We 
use MAG_APER (with 1.5" diameter apertures) for source detection 
and measuring colours, and MAG_AUTO for deriving the physical 
quantities such as star formation rates. Photometric zero points are 
derived from standard star observation of GD71 for NB2071 and 
GD153 for the z'-band, while we scale the J and K s data to match 
the existing data presented in lKodama et al.l fcoOTi). adopting their 
updat ed zero points (see also Tanaka et alj 20101 : IPohertv et al.l 
l2010n . We here apply a small correction to the NB2071 photom- 
etry (by —0.04 mag), based on the median i^s — NB2071 colours 
of bright sources with mNB207i = 17-20 mag. At the position of 



the PKS 1138, we estimate the Gal actic extinction to be E{B — 
V) — 0.04 based on the dust map of lSchlegel. Finkbeiner. & David 
( Il998h . This corresponds to Ab = 0.17 mag, A^, = 0.06 mag, 
Aj = 0.03 mag, Aks = 0.0 15 mag, and Anb207i = 0-01 ma, 
assuming the extinction law of lCardelli. Clavton. & MathisI ( Il98 



2.4 Spitzer MIPS 24fim data 

We use the Spitzer MIPS jRieke et al1l2004t) 24/im data retrieved 
from the Spitzer Heritage Archive. The 24/im filter covers 20.8- 
25.Sfim corresponding to the rest-frame 6.6-8.2 fim for z — 
2.16 galaxies, which is dominated by 7.7/im PAH emission line. 
The retrieved data consist of three AORs (14888704; 14888960; 
14889216) observed under the observing programme 20593, and 
the data covers a ~5' x 5' field around the radio galaxy. We find 
that the post-basic calibration data (PBCD) images created at 
Spitzer Science Center show large-scale sky patterns. To remove 
the artefacts, we apply the self calibration on the basic calibration 
data (BCD) images following the recipe in the MIPS Data Hand- 
book. We then remosaic ed the images using the MOPEX software 
jMakovoz & Khatj2005l) with 1.25" sampling scale. Source detec- 
tion and photometry is performed with SEXTRACTOR. The limit- 
ing flux is measured in the same way as we did for optical and NIR 
data (with 12" diameter aperture). The resultant 3-ct limiting flux is 
39//Jy. We detect 164 sources in total at >3-a significance within 
the 5' X 5' MIPS field, among which we find 15 are associated with 
Hq emitters. This matching is based on the nearest optical/NIR 
counterpart to each source, and we caution that, as a result of the 
poor PSF of 24/im data, these identifications may be uncertain for 
heavily blended sources in some crowded regions. When determin- 
ing the 24-/im fluxes, we use 12" aperture photometry (x 2 of PSF 
size) with an aperture correction of 1.7 x following the MIPS Data 
Handbook. 



^ The photometric zero-points used in^ Kodam a et al.l |2007|) were not ac- 
curate, and we have connected them by the following amounts in the current 
paper; Jcorr = J — 0.25 and i^s.corr = Kg — 0.30. 
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Table 1. A summary of our Subaru imaging data for the PKS 1138 fields. The PSFs are measured on the mosaiced images (they are finally smoothed to 0.7' 
FWHM for multi-band photometry). Note that we re-use the J and Ks data taken in 2006 which were presented in Kodama et al. (2007). 



Field 


Filter 


Instrument 


FoV 


Obs. date 


Exp. time 


Limit mag. 


PSF 












[min] 


(1.5", 5(7) 


FWHM 


PKS 1138 


z' 


Suprime-Cam 


27' X 34' 


2011/4/29 


75 


25.8 


0.7" 


PKS 1138-C 


J 


MOIRCS 


4'x 7' 


2006/1/7, 2011/4/14 


151 


24.5 


0.7" 


PKS 1138-S 


J 


MOIRCS 


4'x 7' 


2011/4/17 


78 


24.2 


0.5" 


PKS 1138-C 


Ks 


MOIRCS 


4'x 7' 


2006/1/6 


55 


23.2 


0.7" 


PKS 1138-S 


Ks 


MOIRCS 


4'x 7' 


2011/4/14, 2011/4/17 


36 


23.2 


0.5" 


PKS 1138-C 


NB2071 


MOIRCS 


4'x 7' 


2011/3/11 


186 


22.9 


0.7" 


PKS 1138-S 


NB2071 


MOIRCS 


4'x 7' 


2011/4/14,17 


119 


22.7 


0.5" 



3 ANALYSIS 

3.1 Selection of Ha emitters 

We select galaxies with a flux excess in the NB2071 filter com- 
pared to the Ks filter. In Fig. (2] we plot — NB2071 colours of 
all the NB-detected sources against NB2071 magnitude. Consid- 
ering the deviation of the plotted points around JCs— NB2071 — 
and the photometric errors, we define NB2071 emitters as 
the galaxies which satisfy both (1) /<s-NB2071 > 0.2 and 
(2) _ft:s-NB2071 > 2.5E (indicated by the solid-line curves in 
Fig|2), where S is the significance of the narrow-band excess 
teunker et alj|l99^ . These criteria correspond to EWrost k, 20 A 
and / (Ha)> 3x 10~^^ erg s~^ cm~^ (which corresponds to adust- 
free star formation rate of ^ lOAfeyr"^; see §3.3). We find 83 
sources satisfying these criteria in total (Fig.|2]l, among which 12 
are fainter than 2-a limit of our Ks-band data. We do not use these 
T^s -undetected sources in the following analyses (except that we 
show their spatial distribution in §4.1), because it is impossible to 
accurately measure their J — Ks colours, stellar masses or star for- 
mation rates. We confirmed that omitting these faint sources does 
not affect our results. 

We checked that all nine spectros copic members at z ~ 2.16 
with visible Hq emission presented bv lKurk et alj j2004bh (except 
for one blended source in our NB image) are indeed selected as 
NB2071 emitters in our analysis, confirming the reliability of our 
emitter selection procedure. In contrast , a majority o f the spectro- 
scopically confirmed Lya emitters ( iPentericci et al.i 2000) do not 
show detectable Ha emission. This is not surprising because the 
nature of Lya emitters and Hq emitters may be different, and this 
is in fact the strong motivation to study distant galaxies using the 
Hq line, which allows us to select analogs of local, star-forming 
galaxies. We also cross-checked with the photometr ic sample of 
Hq emitter candidates shown in iKurk et al.1 j2004al). Out of the 
39 Ha emitter candidates listed in iKurk et alj ( 12004 jfl 20 have 
unique counterparts in our NB-selected catalogue, while the re- 
maining 19 are too faint or blended in our Subaru data. We find that 
18 out of their 20 candidates in our catalogue (90%) show a NB ex- 
cess in our analyses. The mis sing two sources ar e the Hq emitter 
candidates IDS and ID 154 in lKurket all l l2004al) . For ID 154, we 
can still see an excess in our narrow-band data (and the source is 
indeed located near the selection boundary in Fig. [2}. On the other 
hand, the ID5 source shows no narrow-band excess (A's— NB~0) 
in our analysis. The reason for this is not clear, but it may be that 



I ' ' ' I ' ' ' I ' ' ' I ' I ' ' ' I ' ' kll 




17 18 19 20 21 22 23 
NB2071 [mag] 



Figure 2. The A's — NB2071 versus NB2071 colour-magnitude diagram 
to define the NB2071 emitters. The vertical dashed-lines show S-a and 3-(t 
limiting magnitudes in NB2071. while the slanted dotted line shows the 2-cr 
hmiting magnitude in the Ks band. The solid-line curves indicate ±2.52 
excess in A's-NB2071 colours. Galaxies with A'a-NB2071 > 0.2 and 
As— NB2071 > 2.5E are defined as the NB2071 excess sources (black 
symbols). The triangles show emitters with Ks < 2cr, and two are above 
the boundary (shown as arrows). 



3 iKurk et alj [lOOAih listed 40 Ha emitter candidates, while they suggest 
that one of their candidates (1D29) is a low-redshift interloper. We exclude 
this source from our analysis. 



the photometry for this source is less accurate in lKurk et al. I( l2004al) 
because it is located near the edge of their VLT survey field. 

3.2 Reliability of the selection 

The narrow-band excess sources selected above may contain a frac- 
tion of foreground or background contamination. The narrow-band 
technique guarantees an excess of emission at A = 2.07^m in 
the selected NB emitters, but this could be caused by the [Olll] 
line from emitters at z ~ 3.13, [On] emitter s at z ~ 4.55, or 
the PaQ or Pa/3 em itters at lower redshifts (e.g. iGeach et al.ll2008l : 
ISobral et al.ll2012h . To check the reliability of our Hq emitter se- 
lection at z = 2, we show the Bz K diagram for thes e sources in 
Fig.m The BzAT- selection technique jPaddi et alj2004l) is designed 
to select distant galaxies over a relatively broad range in redshift 
(1.4 < z < 2.5), but the major contamination concerned above 
would fall outside this range. Fig. [3] clearly shows that a majority 
of NB2071 emitters satisfy the BzK criteria, suggesting that they 
are really Hq emitters at z = 2.16. Also, most of the NB emitters 
are located in the top-left region of the BzK diagram (i.e. satisfying 
the sBzK criteria), supporting that they are star-forming population. 
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Excluding the sources close to bright stars (for which Biz' photom- 
etry is not possible), we find that 48 out of 56 (85%) NB2071 emit- 
ters detected in both z' and Ks within the B-band data coverage 
satisfy the BzK selection criteria. Among the eight sources which 
fall outside the BzK criteria, four still satisfy the BzK criteria within 
their photometric en'ors. We also check the completeness of the 
BzK criteria using the spectroscopically confirmed members. We 
show in Fig.[3]the BzK colours of 14 spectroscopic members of the 
PKS 11 38 str ucture reported in IPentericci et"Zr i l2000l) . lKurk et al.l 
( l2004bl) , and lDohertv et aL I ( I2OIOI) and detected in our catalogue. 
All the members satisfy the BzK criteria (within the errors) as ex- 
pected, except for one X-ray detected galaxy at 2 = 2.157. This 
should not be surprising as the strong active galactic nucleus (AGN) 
activity in this galaxy affects its broadband colours. 

The BzK selection may not be entirely suitable for exclud- 
ing the high-redshift contamination (i.e. [Olll] emitters at z ~ 3) 
which may be a concern. This would require deep spectroscopy for 
all of the sources to fully understand the contamination rates, but 
this is not available. We therefore check the photometric redshifts 
(photo-z) of our NB emitters derived by Tanaka et al. (201^. Due 
to the lack of multiband photometry in the southern half of our 
field, we can derive photometric redshifts for only 40 bright emit- 
ters (around half of our full Ha emitter sample) located in the north- 
em half of our survey. Although it is not easy to derive accurate 
photo-2:'s for 2 ^ 2 galaxies, particularly for star-forming galax- 
ies with relatively flat spectral energy distributions (SEDs), our 
photometric redshift analysis shows that > 70% of emitters have 
Zphot = 1.8-2.4 (the majority of outliers show Zphot < 1-8), sup- 
porting our claim that a majority of NB 2071 emitters are Ha emit- 
ters al z — 2.16. The contamination rate may of course be higher 
in the southern part of our survey field. Howe ver, a similar depth 
narrow-band study of 2: = 2.23 Ha emitters bv lGeach et alj 120081) 
estimate that the high-redshift contamination (i.e. [Olll] emitters at 
2 ~ 3) into the NB emitters (selected at ^ 2fim) is negligibly small, 
by applying the Lyman Break Galaxy (LBG) technique (they find 
only one source satisfying the LBG selection from 180 emitters 
in their whole sample). Based on this estimate, we expect that the 
high-redshift contamination into our sample would be negligibly 
small. In summary, we estimate the total contamination rate would 
be ^ 10%, even if we consider all non-BzK galaxies as contamina- 
tion. In this paper, we assume all the 83 NB emitters are Ha emit- 
ters at 2 = 2.16, but the results do not change if we apply either 
the BzK or photo-2 selections. 

3.3 AGN contamination 

Recent studies showed that the fractio n of AGN in c reases 
towards high-redshift clusters (e.g. Eastman et al. I I2OO7I ; 



Martini. Sivakoff. & Mulchaevll2009l ; rTomczak. Tran. & Saintongg 
201 ih . Although we expect that the majority of our Ha-selected 
sources are star-forming galaxies, a fraction of Ha emitters might 
be powered by AGNs, and so we here check the possible contri- 
bution of AGNs into our sample. Firstly, we match our sample 
with the Chandra X-ray catalogue ( down to a 0.5-2 keV fl ux of 
~ 10~^^ ergs~^ cm^^) presented by IPentericci et all j2002h . and 
find that five of our Ha emitters are detected in the soft X-ray 
band: the source IDs o f #3, #5, #6, #7 ( =radio galaxy), and #17 
in the nomenclature of IPentericci et al] ( l2002h . We find that all 
of them are known proto- cluster members from the subsequen t 
NIR/optical spectroscopy jKurk et alj l2004bl ; ICroft et all l2()05h . 
and so our survey confirms Ha emission from all of them, while 
we do not find any additional candidate X-ray AGNs. 



M 
(0 

B 



I 




all NB sources spec HAEs (K04/D10)j^ 
NB emitters *: spec LAEs (POO) 



1 2 3 4 5 6 
B-z' [mag] 

Figure 3. The BzK diagram to check the colours of NB2071 emitters. The 
grey dots show all the NB-selected sources which are detected at both z' 
and Ks, within the _B-band data coverage. For _B-undetected sources, we 
replaced their S-band magnitudes with the l-a limiting magnitude, and 
show their lower limits of {B — z') colours with the right-ward arrows. The 
NB2071 emitters are shown with black squares. The red and blue pentagons 
are spectroscopically confirmed Ha emitters (from Kurk et al. 2004b and 
Doherty et al. 2010) and Lya emitters (from Pentericci et al. 2000), respec- 
tively. 



Secondly, we check the SEDs of the Ha e mitters, using 
the be st-fit SEDs derived from the updated code of iTanaka et al.l 
( I2OIOI) . The SED fitting was performed for 40 Ha emitters in the 
PKS 1I38-C field where multi-band data are available. Although 
we are aware of significant uncertainties in our SED fitting, we find 
that most of our Ha emitters are fit by star-forming galaxy tem- 
plates, and that passive SEDs are a minority. We also check the dust 
extinction (Av) estimated from the SED fitting. An accurate esti- 
mate of dust extinction from SED fitting is extremely difficult, due 
to the degeneracies of age, metallicity and dust extinction, but we 
find a weak trend that those Ha emitters with red (J — Ks) colours 
tend to display higher extinction (Av ^ 1 mag), compared to blue 
galaxies (Ay ^ 1 mag). This would also suggest that the red Ha 
emitters are likely to be star forming, rather than dust-free, passive 
galaxies. Therefore, in this paper, we assume all the Ha emitters 
(including five X-ray detected sources) are star forming, although 
we cannot completely rule out the presence of AGNs without deep 
spectroscopy for all of them. 



3.4 Stellar mass 

The transformation from rest-frame luminosity to stellar mass has 
the least dispersion (due to different star formation histories and 
reddening) in the NIR bands. Unfortunately, the Spitzer IRAC (rest- 
frame NIR at 2 = 2.16) coverage of our field is incomplete and so 
we choose to use the more complete Ks-band imaging to estimate 
stellar masses. We confirm that the two techniques agree by com- 
paring the stellar masses derived with the simple Ks-band method 
described below, with those deriv ed from the SED fit ting includ- 
ing Spitzer IRAC photo metry by iTanaka et al. I l l20IOl) (using the 
lBruzual&Charlolll2003l stellar population synthesis model). We 
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find excellent agreement, at least for the brightest sources where 
the SED fitting is possible. Hence, as the SED fitting can only be 
performed for the brighter sources at the northern half of our survey 
(those with IRAC coverage), we use the Jfs-band estimates for all 
our sample in this paper to keep consistency throughout the field. 

The stellar mass estimates from the JsTs-band use the observed 
galaxy luminosity and take into account the variation in mass to 
light ratio (M/L) of the galaxies, dependent upon their star forma- 
tion history (or equivalently their observed colours), by exploiting 
the following equation: 

log(M./lo"M0) = -OA{Ks - 22.24) + A log A/, (1) 

which is derived by simple scaling of t he model galaxies with 
M« = 10"Mq (with Ks = 22.24) from lKodama. Bell. & BoweJ 
( Il999l) , assuming a formation redshift of 2/ = 5. The colour de- 
pendence of mass-to-light ratio (M/L) is included in the last term, 
AlogM = 0.03-1.5 X exp[-l.ll x {z' - Ks)], which is de- 
rived by fitting the model galaxies with a range of star-formation 
histories jKodama. Bell. & Bowei]|l999h . We use [z — Ks) colour 
which neatly straddles the redshifted 4000A break providing a sen- 
sitive tracer of the typical age of the stellar populations in these 
galaxies. Unfortunately we find that there are two Ha emitters near 
a bright star for which accurate optical photometry is not possi- 
ble. For these two sources we instead use the (J — Ks) colour to 
predict the M/L using the same model with AlogM = 0.14- 
0.9 X exp( — 1.23 x ( J — Ks)). We verified that this method pro- 
vides consistent results with those derived using (z' — Ks) colour. 
We also rescale the stellar mass using the ISalpeterl ( Il955l) initial 
mass function (IMF) for consistency with the derivation of SFRs as 
described below. Finally, we correct for the stellar mass account- 
ing for the contribution of Ha emission line flux to the broad-band 
photometry, using our measured NB fluxes. 

Applying different models or IMFs would slightly change the 
absolute measurements of A/* , but the trend we show in this paper 
does not change if we apply different models/IMFs. Indeed, we 
verified that our conclusions do not alter if we apply an empirical 
conversion from Jf-band photometry to A/* (with A'l/L correction 
depending on J — Ks colour) using the SED fitting results for z = 
2.2 Ha emitters from HiZELS (D. Sobr al; private communication) , 
which ass umes the update d version of iBruzual & Chariot! j2003l) 
model and lChabrierl ( |2003|) IMFs. 



3.5 Star formation rate 

Our narrow-band Ha imaging allows us to measure the emission 
line strengths and hence SFRs of all the sources. We derive the 
SFRs by estimating the line flux (-FHa+[Nii])> continuum flux den- 
sity (/c), and the rest-frame equivalent width (EWrost) with the fol- 
lowing equation^: 



EW,est(Ha + [Nil]) 



(1 + ^) 



-1 -fHa + [NII] 



(4) 



Ha + [NII] 



/nb - /Ks 
1 - Anb/Ak, 



fiis ~ /nb(Anb/Aks 
1 - Anb/Aa', 



(2) 



(3) 



^ During this process, we find a source located near the central radio galaxy 
which is estimated to have an unrealistic negative continuum level. It ap- 
pears that the MACAUTO photometry is failing for this source because of 
the extended light from the central radio galaxy. We therefore decided to 
exclude this source from the following analyses where stellar mass or star 
formation rate is needed. 



where A^^ (= 0.31^im) and Anb (= 0.027^im) are the FWHMs 
of the Ks and NB2071 filters, and /k, and /nb are the flux den- 
sities at A's-band and at NB2071, respectively. We then transform 
^Ha+[Nii] to Lhq+[niii- Finally, we compute t he Ha-based star 
formation rat es (SFRhq) usin g the relation of iKennicuttI i ll 9981) 
assuming the ISalpeted ( Il955h IMF; SFRHa(M0 yr"^) = 7.9 x 
IO^'^^Lhq, (erg s~^). We here adopt an empirical correction of the 
[Nil] line contribution to the measured line fluxes using the rela- 
ti on between [Nll]/H a and the rest-frame EWhq+[niii presented 
m ISobral et all ilOlA as wel l as the stellar mass -dependent dust 
extinction correction shown in Garn & Best ( 2010l) after ta king the 
IMF difference into account jGarn & Beslll2010l assumed iKroupal 
I2OOII IMF, and so we scaled their stellar mass by -1-0.24 dex). We 
checked that most of our results are unchanged even if we applied 
the conventional consta nt I -mag correcti on or a SFR-dependent ex- 
tinction correction (e.g. lOam et al.l2oTol) . but some possible effects 
on our results will be discussed in §4.4.4. 

For the MlPS-detected emitters, we also estimate the IR- 
derived star formation rates (SFRir) using the 24/im photome- 
try. We first estimate the total infrared luminosity (Lm ) using the 
model SEDs of starburst galaxies jLagache. Dole. & Pug et 2003|), 
and then transform to SFRir using the IKennicuttI l l 1 998l) relation; 
SFRiR (Mgyr-^) = 4.5 x 10"'''*Lir (erg s"^). We find that the 
MlPS-detected emitters have SFRir ;iIOOMoyr"\ which are 
typically factor ~ 2-3 higher than the Ha-derived SFR. Therefore 
it is possible that the A/, -dependent extinction correction applied 
above may be still underestimating the extinction correction in the 
massive galaxies (note that the MlPS-detected emitters are most 
massive emitters with Af« ^ 10^^ Mq; see §4.2). In this paper, we 
use the Ha-derived SFRs, due to the limited number of the MlPS- 
detected emitters and the restricted MIPS data coverage, and due 
to the large uncertainty in de riving Lir from a s ingle rest-frame 
~8/^m photometric point (e.g. lNordon et alj20IOl) . 



4 RESULTS AND DISCUSSIONS 

4.1 Large-scale structure around PKS 1138 

We show in Fig.|4]the spatial distribution of the Ha emitters as well 
as the MIPS and X-ray detected emitters. It is clear that the Ha 
emitter candidates are highly concentrated around the radio galaxy, 
which is consisten t with the previous Ha study of this field by 
lKurketaljj2004al) . In contrast, the number density of Ha emitters 
in the southern part of the field is significantly lower. To quantify 
this we estimate density of Ha emitters in the proto-cluster core re- 
gion (within 40 arcsec from the radio galaxy) as ~ 10.0 arcmin^^, 
which is ~ 10 times higher than the average number density calcu- 
lated in the southern half of our survey. Thus, our findings confirm 
the many previous studies which have claimed the presence of an 
over-density of galaxies in the PKS 1138 field from various tech- 
niques (e.g. IPentericci et al.ll2000l; iKurk et alj|2004d ; ICroft et al.l 
l2005l ; lKodama et al.ll2007l ; lTanaka et al.ll2010l) . 

We also see from Fig.|4]that the PKS 1138 proto-cluster is ac- 
tually embedded in a very large-scale filament, running from north- 
east to south-west through the cl uster centre . The n orth-east fila- 
ment has already been reported bv lKurk et al. I( l2004ah , but our new 
survey has revealed an even larger-scale filament (> 10 Mpc in co- 
moving scale) traced by Ha emitters, which also possibly extends 
off to the south east (although, unfortunately, masking due to bright 



© 0000 RAS, MNRAS 000, 000-000 



Massive starbursts ina z ^ 2 proto-cluster 7 



AR.A. [Mpc (comoving)] 




2 1 0-1-2 
AR.A. [arcmin] 



m 

< 2 

do 

<a 



I I I I I I I I I I I I I I I I ^ I M I I I I I ii 
Prolo-clusler(PKS1138)\^\ 1 1 

- H ....s<=.i.o. 



1 -■•■z,=3 



I 



m 

< 2 



00 

CO 



EQ 

I 



•isi 



\ \ I I 

■»,..I..\..J., 

N ^1 



m' 1 1 1 1 1 1 r r I I ' l 1 1 1 i M 1 1 1 1 1 ii 




Low-density fields 

(PKS1138-S + GOODS-N) 

z,= 10 



"■■(■■■;;;■■■>,. J..\.. J., 



z,'='5 

M*=10"Mii.-;'-^ 



■ I I I I I •■ I 'I I I I I I ■ T I ■ I I ■ I 



I "... H 



I . . I I 




8 19 20 21 22 23 24 
Ks [mag, AB] 



1 10 

N 



Figure 4. {Left): The 2-D map of the Ha emitters around PKS 1138. The black squares show Ha emitters which satisfy BzK selection, while black circles 
show Ha emitters which do not satisfy the BzK selection. The triangles show the -undetected emitters. The red filled circles and black open squares show 
MIPS and X-ray detected emitters, respectively. Grey points show all the NB2071 -selected sources, and DRGs are marked with red open circles. Note that 
the B-band data is only available at ADec.>— 3.5, so that we do not have BzK information below this line. The coordinates are given relative to the central 
radio galaxy. The large grey circle shows the masked region due to some bright sources. The dotted line at ADec.=— 2.0 shows the dividing line of high- and 
low-density environment used in this paper (Right): The (J — Ks) vs Ks colour-magnitude diagram for the high-density proto-cluster and the low-density 
fields. The grey dots indicate all the NB-selected sources, and the black squares are Ha emitters (red filled circles for MlPS-detected emitters). The diamonds 
in the bottom panel are the z = 2.2 Ha emitters in the GOODS-N field. The vertical and slanted dashed lines s how 3-cr and 2-a limiting m agnitudes of 
Ks-band and J-band data, respectively. The locations of the red sequence in the case of = 3, 5, 10 modeled bv lKodama & Ari motdi 19 971) are shown a s 
the dotted fines, and the dash-dotted fine shows the iso-stellar-mass line (corresponding to IO^^AIq) based on the model of lKodama, Bell. & BoweJ fl999l) . 
The colour distribution of emitters within each environment is shown as the histogram, where we also show our colour definitions. 



objects is hiding some of the structure). Such large-scale structures 
in the nearby Universe are generally traced by passive galaxies, 
but the situation is not necessarily the same in the distant Uni- 
vese, where star-forming population are also highly clustered in the 
proto-clus ter core (se e consistent res ults from 1.5 < z < 2.5 cluste r 
studies bv lHavashi et al. 2010; Hilto n et alj2010l : lTran et alj2010h . 
Nevertheless, the structure around the PKS 1138 is also suggested 
from a n analysis of distant red galaxies (DRGs) bv lKodama et al.l 
( |2007|) . although the structure traced by Hq emitters presented in 
this study appears to be much more prominent. This may be be- 
cause the DRG sele ction picks up di stant galaxies over a wide 
range in redshift (e.g. lFranx et alj2003l) . diluting the contrast of any 
structures. We check the spatial distribution of DRGs throughout 
the field (Fig.|4l and confirm that the spatial distribution of DRGs 
is qualitatively consistent with that of Ha emitters, but again, the 
structure looks less prominent. 



4.2 Red star-forming galaxies and their environment 

4.2.1 Colour-magnitude diagram 

We show in Fig. |4] the colour-magnitude diagrams for high- 
and low-density environment separately. Considering the complex 
structure around PKS 1138, we simply define the upper and lower 
half of our survey field as high- and low-density region, respec- 
tively (see Fig.|4l(. Because the number of Ha emitters in the low- 
density environment is much smaller than that in the high-density 
region, we included in our analysis the z — 2.2 Ha emitter sample 



fr om the blank field su rvey in the GOODS-North field with Subaru 
bvlTadakietalJ ( l201ll) . Although the NB filter used in lXadaki et all 
bOllh is different (NB209 filter with Ac = 2.09^m), the Ha emit- 
ter selection technique and all the broad-band filters used in that 
study are exactly the same as those used in this study. After apply- 
ing the same flux and EW cut, we add these sources in our field 
galaxy sample. 

In Fig. |4l we find a large number of "red Ha emitters" with 
(J — Ks) Z 1. In some extreme cases, they satisfy the DRG selec- 
tion criteria with (J— A's) > 1.38 (equivalently, ( J — A"s)(vcga) > 
2.3). We confirmed that the two r ed sources with Ha emission 
shown in the spectroscopic study bv lDohertv et al.l ( 1201 Oh are both 
selected as red emitters in this study. Importantly, we find that such 
red Ha emitters are predominantly found in the proto-cluster envi- 
ronment, while they are very rare in the low-density environment. A 
Kormogorov-Smirnov test (KS test) shows that the probability that 
the colour distribution of the high-density and low-density samples 
are drawn from the same parent population is only 0.1%. We note 
that this result may partly be produced by the presence of X-ray de- 
tected AGNs which also tend to be red and massive (see Fig.|4ll, but 
the result does not change even if we exclude all the X-ray sources. 

We also find that the red Ha emitters tend to have system- 
atically brighter A's -band luminosities compared to blue emit- 
ters. We show in Fig. |4] the iso-stellar-mass line corresponding 
to Mt = 10^^ M0 based on the same model as that used when 
deriving M, (see §3.4). It is notable that the Ha emitters with 
red (J — Ks) colours tend to have very high stellar masses with 
Mt k, IO^^Mq. These massive star-forming population are ab- 



© 0000 RAS, MNRAS 000, 000-000 



8 Y. Koyama et al. 



AR.A. [Mpc (comoving)] 
4 2 -2 -4 



o - 



T" 



T — I — I — I — I — r 



"T 1 1 1 1 1 1 1 1 1 1- 



•••• ■ □* ^ □ 



4' ' \ 




J ' ' ' ' ' 1—1 ' ' ■ ' ' L_l_l ' ' ■ ' ' L_I ' ' ■ ' ' L_I L 



ilk 



HO !> 

> 
o 

7 o 
o 



o 

(U 

CO Q 

I < 



2 1 0-1-2-3 -4 
AR.A. [arcmin] 



ft 2 



M 
QO 
O 



1 1 1 1 1 1 1 1 1 1 

"^^^^^^ 

■ ^1 1 ijS^ 


1 1 1 1 1 1 

Jl 


red HAE].^^'^'"^^" 
■|>l"*T . 1 . . . 1 . . . 


HAEy : 

blue HAE J 

. . . 1 . . " 



^ 13 F" 
2° 12 1- 

>nr 

"aS 10 
^ 9 



2 4 6 8 

Distance from central RG [Mpc] 

— I 1 1 p — I 1 1 r 



T 1 1 — 



Jl 



_l I I I L- 



1 2 



Figure 5. (Left): The same plot as Fig. 4, but using different colour symbols based on their (J — Ks) colour. The red, green and blue squares indicate Ha 
emitters with (J - Ks) > 1.38 (DRGs), 0.8 < ( J - Ks) < 1.38 and (J - Ks) < 0.8, respectively (see text). The black triangles are the A's -undetected 
emitters. (Top-right): The cumulative fraction and the local number densities (calculated with all emitters) of red, green, and blue emitters as a function of 
distance from the central radio galaxy. The different colour symbols indicate different (J — Ks) colours, and the open squares show the X-ray detected 
emitters. (Bottom-right): The stellar mass of emitters as a function of the local density. The meanings of the symbols are the same as those in the top panel. 



The vertical dashed line shows the median value of logj^Q S5 



1.04. 



sent from lower-redshift clusters, and therefore we propose that 
they are the progenitors of present-day passive cluster galaxies (see 
more detailed discussion in §4.4). Our result suggests that the clus- 
ter galaxies had already formed a large part of their stellar mass 
by z ^ 2, but they are still in a vigorously star-forming phase. 
There is no strict definition of "green" galaxies, but we here dis- 
tinguish relatively redder emitters with (J — Ks) ~ 1 (which 
tend to be brighter in Ks-band) from the very blue emitters with 
(J — Ks) ^ 0.5 (which tend to be fainter in Ks-band), as recog- 
nized on the colour-magnitude diagram. Hence, in the remainder 
of this paper, we define the red, green and blue emitters as those 
having (J - Ks) > 1.38 (i.e. DRG), 0.8 < ( J - Ks) < 1.38 and 
(J — Ks) < 0.8, respectively. 

We also show in the colour-magnitude diagram the MIPS- 
detected Ha emitters (red circles in the top panel). Although the 
MIPS data only covers the higher-density region, it is notable that 
most of the MlPS-detected emitters are red/green massive sources. 
Within the MIPS data coverage, we find that 14 out of 29 (48%) 
are detected at 24^m for red/green emitters, while only 1 out of 24 
(4%) is detected for blue emitter. This trend is of course related to 
the difference in the stellar mass distribution between red and blue 
sample (see also §4.4), but this result suggests that the red emitters 
are likely dusty sources rather than passive galaxies. Their MIR- 
derived SFR is estimated to be SFRir > 100 Mq yr"^ (and with 
the dust extinction estimated to be Aua ~ 2-3 mag), suggesting 
that these ULIRG-class active galaxies are a c ommon population i n 
proto-cluster environment at z ~ 2 (see also iTanaka et aljbol ih . 
We note that the red Ha sources without MIR detection also have 
strong Ha emission with SFRhq ^SOAfoyr"^, and so we ex- 
pect that t hey are also strong starbursts rather than passive sources. 
Recently, iMavo et al.l l l2012h analysed the Spitzer MIPS data of a 
number of high-redshift radio galaxy fields including the PKS 1138 



field. That study found an overdensity of MIPS 24/.tm sources in the 
PKS 1138 field at 43-a significance, but it has been impossible to 
study the MIR detection from individual cluster galaxies because 
of the difficulty of determining their membership. We note that the 
clean Ha emitter sample presented here has allowed us to confirm 
the presence of MIR-bright sources in the PKS 1138 proto-cluster. 



4.2.2 Red star forming galaxies: a proto-cluster phenomenon? 

In Fig. [5] we show the spatial distribution of the red, green, and 
blue emitters defined above. This plot demonstrates our important 
finding that the redder sources are strongly clustered in the proto- 
cluster field. We here apply a more quantitative measurement of the 
environment to show the environment of the red massive Ha emit- 
ters at jz = 2. At first, we plot in Fig. |5] (top-right panel) the radial 
distribution of red, green, and blue Ha emitters from the central 
radio galaxy. Using a Kolmogorov-Smirnov (KS) test, we find that 
the probability that red/green Ha emitters and blue Ha emitters are 
drawn from the same parent population is 4.6%. We also find that 5 
out of 10 red Ha emitters (50%) are clustered within 1 arcmin from 
the radio galaxy, while only 9 out of 39 blue Ha emitters (23%) are 
located in the same region, supporting an excess of red emitters in 
the proto-cluster environment. Next we calculate the local number 
density of Ha emitters (E5) and we plot them as a function of the 
distance from the radio galaxy (middle-right panel in Fig.|5](. With 
this plot, we can conclude the over-density of Ha emitters around 
PKS 1138 (by a factor of ~ 10 compared to its surrounding field; 
see also §4.1). We also show in Fig. [5] (bottom-right) a plot of the 
stellar mass of Ha emitters as a function of the local density. Al- 
though the trend seems to be less prominent, we can still see a hint 
that redder Ha emitters prefer the higher-density environment. In 
fact, by splitting the sample into high- and low-density bins at the 
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Figure 6. The HST ACS /814-band snapshots (4" x4" for each) of the Ha emitters in the PKS 1138 field, plotted on the colour versus stellar mass plane (the 
IDs are given based on our Ha emitter catalogue). Note that the locations of each source in this plot does not exactly reflect their stellar masses or colours in 
order to avoid overlap with each other. The colours of border lines of each snapshot indicate red, green, and blue emitters, as we used throughout the paper 
The "X" and "M" denote the X-ray and MIPS detected sources, respectively. Note that we find five X-ray detected Ha emitters, but one of them is outside 
of the HST data coverage. We also plot the Gini coefficients (G) of these galaxies as a function of stellar mass (the top-right panel). The red, green and blue 
symbols indicate their (J — Ks) colours, and the open squares are for X-ray detected emitters. Note that we do not calculate G for ID69 which is too faint in 
the HST image. 



median value of the E5 (shown as the dashed line in Fig. |5}, we 
find that 8 out of 10 red emitters (80%) are located in the higher- 
density bin, supporting an excess of red, massive emitters in the 
proto-cluster environment. Note that as this density measurement 
is made using only the Ha emitter sample (we do not know the 
location of passive galaxies in the proto-cluster), we have decided 
to retain the more simple definition of environment for the remain- 
der of this study (i.e. the high-density environment is defined as the 
northern half of our survey). The results shown here do not change 
if we use the density-based definition. 

Our finding of an over-density of red Ha sources near the 
proto-cluster core is qualitatively consistent with o ur recent study 
of a p roposed proto-cluster field at z — 2.53 by iHavashi et al.l 
( |2012|) . who also found a concentration of red Hq emitters in high- 
density clumps. However, it would be interesting to note that the 
very massive (M* ^ lO^^M©) red emitters are not a common 
population in that study. Such very massive red star-forming galax- 
ies are also very rare in another p roto-cluster field 4C-I- 10.48 at 
Hatch et all 1 201ll) . The colour- magnitude 



= 2.35 studie d by 



diagram shown in iHatch etal.l ( l201lh also suggests that red emit- 
ters themselves are very rare in the 4C-I- 10.48 field, and in fact 
they suggested that there is no strong environmental variation in 
the colour distribution of Ha emitters at 2 ~ 2. These differ- 
ent results between the proto-cluster fields would indicate that the 
cluster-to-cluster variation is significant in the early Universe. In- 



terpretation of this difference is difficult at this point, but it may be 
that the properties of galaxies are related to the richness of clus- 
ters or surrounding structure in the sense that more mature clusters 
tend to harbour redder and more massive galaxies inside them. Per- 
haps, the PKS 1 138 field may be an unusually rich field at 2: ~ 2, 
as suggested by the huge surrounding structure, which may have 
then collapsed early on. In any case, it is clearly important to study 
larger sample of 2 > 2 proto-cluster galaxies in the future to obtain 
firmer conclusions on environmental variations of galaxy proper- 
ties at such high redshifts. 



4.3 HST morphologies 

We check the Hubble Space Telescope (HST ) morphologies of the 
Ha emitters using the ACS /814-band image. Studying morpholo- 
gies of galaxies within proto-cluster environments would be an in- 
teresting approach to understand how the morphology-density rela- 
tion seen in the local Universe has been built-up, but it is generally 
not easy due to e.g. the difficulty of determining cluster member- 
ship (see pioneering works by e.g.lPeter et al.l2007l : |PaDovich et al.l 
kal l2012t) ! 



I2012I : Iz irm. Toft. & Tanak We use the pipeline reduced 

data, whic h have been retrieve d from the Hubble Legacy Archive, 
as used in lTanaka et al. U2OI0I) . Note that the J8i4-band traces the 
rest-frame NUV light for 2 ~ 2 galaxies, and therefore it does not 
necessarily represent their stellar mass distribution. However, our 
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Figure 7. (a): Star formation rates of Ha emitters plotted against their stellar masses. The colour symbols show the different (J — Ks) colours as indicated. 
The grey points are in the case of no dust extinction connection, and the open squares indicate X-ray detected emitters. We also show the "main sequence" of 
2 ~ 2 galaxies from Daddi et al. (2007) and Santini et al. (2009). The histograms in the top and right panels show the Alt and SFR distribution of each colour 
population. Note that we excluded the radio galaxy in this plot due to the large uncertainty in deriving its A/* and SFR. (b): The same plot but changing the 
symbol type based on the environment. Note that the field galaxies include the southern half of the PKS 1138 and GOODS-N field from Tadaki et al. (201 1). 
We also show the Ha-selected galaxies in the z = 0.8 cluster from Koyama et al. (2010) as a comparison, applying an average flux connection (x 1.4) taking 
their NB filter incompleteness into account (see Koyama et al. 2010). The dotted lines show the best-fit main sequence for our z = 2.2 and z = 0.8 data, 
calculated using galaxies with A/» > IO^^Mq. 



motivation here is to understand whether the rest-frame UV light 
is centrally concentrated (dominated by a point source) or spa- 
tially extended, which could also be a crude test of the presence 
of AGNs. The HST data covers 5.5' x 3.5' around the PKS 1138, 
and 54 emitters are located within this HST data coverage (but 
one of them is on a bleed from a bright star). In Fig.|6l we show 
the postage stamps of the available /8i4-band morphologies of our 
(A's -detected) Ha emitters on the colour versus stellar-mass plane. 
Fig. [6] clearly shows that a large fraction of the Ha emitters dis- 
play irregular/clumpy morphologies with spatially extended diffuse 
light, and this trend may be stronger for massive red/green emitters. 
We attempt to quantify this trend by calculating the "Gini" coeffi- 
cient (G) of the Ha emitters, using the selected pixels in the seg- 
mentation map created by S EXTRACTOR (we use the pixels within 
3" x3" region around each source to minimize the effect of nearby 
sources). In Fig. [6] we plot the G value as a function of stellar mass, 
and find that there is a weak trend that red/green massive emitters 
tend to have smaller G, hence more uniform light distributions. We 
caution that the low-mass blue galaxies may be (intrinsically) too 
small to be resolved, which may produce larger G values. Also, the 
smaller G values for red/green galaxies may be because the central, 
nucleated starburst activity is hidden by dust (but we need a direct 
identification of this using e.g. high-resolution sub-millimeter ob- 
servation with ALMA). 

The clumpy morphologies may not be surprising because 
the rest-frame UV morphologies tend to be biased to highly star- 
forming regions. Nevertheless, these clumpy morphologies would 
suggest that the presence of extended star-forming regions within 
those galaxies, and therefore it is unlikely that a majority of the Ha 
emitters are dominated by pure AGNs. In fact, the X-ray detected 
Ha emitters (ID=5, 82, 83, except for the central radio galaxy) 
show relatively compact morphologies compared to the others (and 



tend to have larger G values; see open squares in the top-right panel 
in Fig.|6l(. Although galaxies with such compact UV morphologies 
are not the dominant population in our sample, there are a few more 
galaxies with large G values at the high mass end. These galaxies 
may be additional AGN candidates, but it is impossible to discrim- 
inate between nucleated starbursts and AGNs at this moment. We 
can also notice that the clumpy morphologies may be most promi- 
nent in green emitters. Most of the green emitters (except for the 
X-ray detected ones) have complex morphologies, and many of 
them seem to accompany multiple cores and/or extended diffuse 
light. This could suggest that their activity are triggered by merger 
events, but firm conclusion require rest-frame optical morpholo- 
gies, which will be less biased to star-forming regions. Note that 
a fraction of massive red/green emitters are very faint in the HST 
image (see e.g. ID 52, 50, 44, 26). We can still see some diffuse 
UV emission from these galaxies, and so we expect that their UV 
light are heavil y obscured by du st. Indeed, a recent MIR spectro- 
scopic study bv lOgle et al. I( l2012h shows prominent PAH emissions 
in the rest-frame MIR spectra of ID 44 and 52 (corresponding to 
their HAE131 and HAE229, respectively), which is another ev- 
idence that these galaxies have dusty star formation within those 
galaxies. 



4.4 Star formation "Main Sequence" in cluster environment 

4.4.1 Colour dependence on the SFR vs. A/* plane 

Recent studies have suggested a correlation between stellar mass 
and SFRs of distant star-forming galaxies in t he sense that mas- 
sive galaxies tend to have higher SFR s (e.g. lElbaz et af I l2007l : 
lDaddietal.ll2007l : iKaiisawa et al]|201(]|) . the so-called "Main Se- 
quence" of star-forming galaxies. Using the large sample of star- 



© 0000 RAS, MNRAS 000, 000-000 



Massive starbursts in a z ^ 2 proto-cluster 1 1 



forming galaxies in a proto-cluster environment from our study, we 
test if the proto-cluster galaxies are on the same correlation as re- 
ported for field galaxies at 2: ~ 2. We plot in Fig.|7^ the SFRs of Ha 
emitters in the PKS 1138 field against their stellar mass. We also 
show the main sequence of g ~ 2 galax ies reported in the literature 
jPaddi et al]|2007l ; [Saiitini et alj|2009l) for comparison. Although 
the main sequence of field galaxies are slightly different between 
the various literature studies (depending on the sample selection or 
derivation of physical quantities), our Hq emitters are broadly con- 
sistent with the literature work. We find that the significant outliers 
tend to be the X-ray detected sources at the massive end. For these 
galaxies, AGNs will contribute to the total Ha emission and/or con- 
tinuum level, so that their SFRs and/or stellar masses may not be 
accurate. It is clear that the redder Ha emitters tend to dominate the 
massive end of the main sequence (i.e. higher SFR and masses than 
blue population). Although we cannot see a significant difference 
in SFR between red and blue population, it is possible that the red 
emitters have even higher SFRs, because we may be still underes- 
timating the dust extinction correction in the most dusty systems, 
detected at the high mass end (see §4.4.4). A caveat on this plot 
is that the lack of low-mass red sources is partly due to the J-band 
flux limit (see the colour-magnitude diagram in Fig.|4](, but the lack 
of blue massive galaxies would be a real phenomenon. 



4.4.2 Environmental dependence on the SFR vs. M, plane 

In Fig.|7j3, we show the SFR versus AI, plot, but this time splitting 
the sample into two based on th eir environment. Ag ain, we add 
the GOODS-N Ha emitters from iTadaki et alj 1 I2OI li to the low- 
density sample to improve the statistics. We find that the galaxies 
in the high- and low-density environments show the same main se- 
quence, but with the massive end of the main sequence being dom- 
inated by galaxies from the higher-density regions of the proto- 
cluster. The histograms show the Af, distribution for each envi- 
ronment (i.e. stellar mass function of star-forming galaxies), con- 
firming the trend that massive galaxies are more numerous in the 
higher-density proto-cluster environment. A KS test shows that the 
probability that these two distributions are from the same parent 
population is only 0.5%. We also show the SFR distribution in each 
environment in the right-hand panel of Fig.|7j), showing a possible 
trend that the cluster galaxies have higher SFRs compared to field 
galaxies at 2: ~ 2. This would be a natural consequence of the fact 
that the massive emitters are more numerous in the cluster envi- 
ronment, and again, the trend would be even stronger because we 
may be underestimating the extinction correction for these dusty 
galaxies (see discussion in §4.4.4). Finally, we note that there are 
two sources located significantly above the main sequence in our 
z ~ 2 cluster sample, but these galaxies are X-ray detected AGNs 
(as mentioned in §4.4.1). Again, our conclusion does not change 
even if we exclude all the X-ray detected sources. Therefore we 
conclude that there is no detectable difference between the main 
sequence in the proto-cluster and general field environment, while 
the stellar mass distribution within the sequence shows a significant 
environmental dependence. 



4.4.3 The evolving main sequence in distant cluster environment 

It is also interesting to test the redshift evolution of the main se- 
quence in cluster environments. In Fig[7j3, we plot the distribution 
of the SFR a nd stellar masses for H a-selected galaxies in a 2 ~ 0.8 
cluster from iKovama et all ilOldt . We see a significant difference 



in the main sequence between z ~ 0.8 and z ~ 2.2 sample (a 
factor of ~ 5), which is roughly consistent with the specific SFR 
decline for field galaxies rep orted in the literature (e.g. lKarim et al.l 



I2OI ll: IWhitaker et al.ll2012f) . We note that the z ~ 0.8 Ha study 



bv lKovama et alj ilOld) covers a wide range in environment from 
the very rich cluster core to its surrounding field, which allows us 
to split the sample into two environmental bins to test the environ- 
mental dependence at 2 ~ 0.8. Fig.lTJ) shows that the galaxies at 
2 ~ 0.8 in high- and low-density environments galaxies broadly 
follow the same general sequence, but there may be some interest- 
ing hints on the galaxy evolution in different environment. 

Firstly, we notice that the massive star-forming galaxies with 
Mt ^ lO^^i\'f0 are not present in the 2 ~ 0.8 cluster. We recall 
that the massive end of the main sequence is dominated by galax- 
ies in high-density environment at 2 ~ 2, but the massive end of 
the 2 '-^ 0.8 main sequence is not dominated by galaxies in high- 
density environments. This result would suggest that star-forming 
activity of the massive star-forming population found in 2 ~ 2 
proto-cluster environments have been quenched over the interven- 
ing ~ 3 Gyrs (1 ^ z <^ 2), so that they cannot be identified as star- 
forming at 2 ~ 0.8 any more, confirming th e "down-sizing" ev olu- 
tion of galaxies in cluster environment (e.g. lTanaka et aflboOSl) . 

Another interesting hint we can obtain from Fig.|7]is that there 
seems to be a small offset (a factor of ~ 1.5-2) between the main 
sequence of high-density and low-density galaxies at 2 ~ 0.8. 
We find that > 80% of the 2 ~ 0.8 high-density sample are lo- 
cated below the best-fit relation calculated for the whole 2 ~ 0.8 
sample (see Fig.lTJj). Although the significance is not great (1.7- 
a level), this may be an interesting implication that galaxies have 
evolved more rapidly in high-density, cluster environments than 
in the lower-densi ty field environment at 1 ^ 2 2 (see also 
iTadaki et alj|201 ih . In fact, the lower specific SFR of galaxies in 
high -density environments is consistent wi th some literature works 
(e.g. IPatel et alj|2009l ; IVulcani et al]|2010l) . but the environmental 
dep endence of star-forming activity at 2 ~ 1 is still under debate 
(e.g. lElbaz et al.l2007l : lMuzzin et alj2012l) . We caution that the def- 
inition of environment is not exactly the same between our 2 ~ 0.8 
and 2 ^ 2.2 samples (we divide the 2 ~ 0.8 sample into clus- 
ter/field environment at log S5 = 2 using the local numb er den- 
sity of photo-2 selected galaxies; see iKovama et al.ll2Q07l ; 2008). 
Also, our result is based on just one cluster at each redshift, and 
that these two clusters may not be necessarily on the same evolu- 
tionary track (i.e. it is not clear if PKS 1 138 is really the progenitor 
of our 2 ~ 0.8 cluster). It is clearly important to verify this trend 
based using larger, unbiased sample of galaxies at different red- 
shifts, but Fig.|7]may be giving us a hint of the role of environment 
in the evolution of galaxies at high redshifts. 



4.4.4 A caveat on the effect of dust extinction correction 

Our survey suggests that M, distribution of star-forming galaxies 
is likely to depend on the environment at 2 '-^ 2, in the sense that 
massive star-forming galaxies are more numerous in high-density 
environments. This result is qualitatively consistent with pioneering 
work which investigated the environmental dependence of galaxy 
properties at 2 ~ 2 -3 (e.g. [steidel et al. 2005; Hatch et al. 201j|; 
iMatsuda et al.ll201ll) . We also obtained an interesting hint that the 
2 ~ 2 galaxies in high-density regions have higher SFR than galax- 
ies in lower-density environments at the same redshift, but the SFR- 
related result should be treated with caution because it may be af- 
fected by the uncertainty in the dust extinction correction. We recall 
that we adopted a stellar-mass dependent extinction correction for 
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the Hq luminosities (see §3.3.2). Varying this correction has little 
impact on the stellar mass measurements, but we here discuss some 
possible effects on the SFR-related results. 

As a check of the influence of the reddening corrections, we 
apply (1) the conventional 1 -magnitude correction for dust red- 
dening in Hq samples and (2) the SFR-dependent correction pre- 
sented bv lGarn et al.l ( |2010|) . We find that these methods both make 
the main sequence slope flatter, so that the environmental depen- 
dence in SFR is likely to be washed out. This is in fact consistent 
with some earlier works which reported no environmental varia- 
tions in SFR distribution (i.e. the shape of Ha luminosity functions 
rather than its normalisation or characteristic lum i nosity) among 



star-f orming galaxies at 2; ~ 2 jHatch et"al] 



2OIII: iMatsuda et alj 



Koyama et al 



20 id : 



20 111: see also sim ilar results at z ~ 1 by 

Sobral et"al] 1201 lb . However, as far as we believe the main se- 
quence reported in the literature, the constant 1 -magnitude correc- 
tion tends to underestimate the extinction of massive galaxies, in- 
deed we estimate the extinction of such massive Ha emitters to 
be Ana — 2-3 mag based on the MIR photometry. This is even 
higher than that estimated using the -dependent correction we 
adopt in this study, although the SFR deri ved from rest-frame 8/im 
photometry might be overestimated (e.g. iNordon et al.ll2O10l) . On 
the other hand, the SFR-dependent extinction correction seems to 
be overestimating the extinction of low-mass galaxies (again, if we 
assume that they are the normal, main-sequence galaxies). There- 
fore, we expect that the M* -dependent correction adopted in this 
study would be more realistic at this stage, although it is currently 
impossible to accurately measure the extinction effect on individual 
galaxies. 



5 SUMMARY AND CONCLUSIONS 

We have presented a panoramic survey for Ha emitters in the proto- 
cluster around PKS 1138 al z = 2.16 using a narrow-band fil- 
ter with MOIRCS on Subaru. We study properties of star-forming 
galaxies in the proto-cluster, and discuss their environmental de- 
pendence. Our results are summarized as follows. 

(1) A prominent large-scale structure is identified around 
PKS 1138. We find that the PKS 1138 proto-cluster is embedded 
in a > 10 Mpc-scale filament traced by Ha emitters and extending 
from north-east to south-west through the radio galaxy. We also 
find a possible filament running towards south-east from the ra- 
dio galaxy, suggesting that the PKS 1138 field is located at a node 
of the cosmic web at z = 2.16. Several emitters are likely to be 
AGN (five are X-ray detected), but we expect that the majority of 
our sample are star-forming galaxies, based on the SED analysis 
and on their clumpy/extended rest-frame UV morphologies in HST 
imaging. 

(2) Galaxies in the z — 2 proto-cluster environment tend to 
have redder colours and higher stellar masses (and probably larger 
SFRs) compared to those in underdense regions. In particular, we 
find that a substantial fraction of Ha emitters in PKS 1138 show red 
colours with (J — Ks ) ^ 1, and in some extreme cases, they satisfy 
the DRG criteria. Such red Ha emitters tend to have very large 
stellar masses (Al, > 10^^ A/q), compared to the more normal blue 
galaxies. This is an evidence that while many cluster galaxies are 
still in a vigorously star-forming phase, they have already formed a 
large part of their stellar mass by 2 ~ 2. 

(3) The red Ha emitters tend to be detected in the Spitzer 
MIPS 24/im imaging, suggesting they are luminous and dusty. 
Within the MIPS data coverage, we find that 14 out of 29 (48%) 



are detected at 24^m from our photometrically-defined sample of 
red/green emitters, while only 1 out of 24 (4%) is detected for 
the blue emitter sample. Such 24/im-detected Ha emitters are es- 
timated to have SFRm ;> lOOMeyr"^ (i.e. ULIRG class) and 
heavy dust extinction with Aua ~ 2-3 mag. Therefore we sug- 
gest that these intense starbursts are a common population in the 
young proto-cluster environment, while they are absent from the 
cores of lower redshift cluster at 2 ^ 1. 

(4) The red Ha emitters may be a cluster phenomenon. Our 
survey reveals that the red Ha emitters are preferentially found in 
the higher-density proto-cluster region, while the blue Ha emitters 
are more widely spread. Therefore, we expect that the red, massive 
star-forming galaxies are the key population driven by environmen- 
tal effects in the early Universe, which go onto become present-day 
passive cluster galaxies. 

(5) The Ha emitters in proto-cluster environments are on 
broadly the same "main sequence" of 2 ~ 2 star-forming galaxies 
as those in the field, but there is an excess of massive systems com- 
pared to lower-density environments. In contrast, this trend may 
not be present at 2 ~ 1, where the most massive galaxies in high- 
density regions are not forming stars any more. We also find a ten- 
tative hint that the star-forming galaxies in cluster environments at 
z ~ 0.8 may have lower star-formation activity than in field at the 
same redshift, implying that the accelerated galaxy evolution we 
see is associated with the cluster environment at 1 < 2 < 2. 
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